Template-Based Conformal Shape-from-Motion from Registered Laparoscopic Images

One of the current limits of laparosurgery is the absence of a 3D vision facility for standard laparoscopes. While there has been significant progress made in visual SLAM (Simultaneous Localization And Mapping) with a single camera, most of the current approaches relies on the assumption that the tissues are rigid or undergo a cyclic deformation. However, in laparoscopic surgery none of these assumptions commonly apply, due to unpredictable and non-isometric deformation of the tissues. Our main contribution in this paper is a new sequential 3D reconstruction method well-adapted to the peritoneal environment. We draw on recent computer vision results exploiting a template of the environment. The state-of-the-art methods assume that the environment can be modeled as an isometric developable surface, i.e. which deforms isometrically to a plane. While this assumption holds for paper-like surfaces, it certainly does not fit for peritoneal surfaces. Our new method tackle these limits: it uses a full 3D template and copes with non-isometric 3D deformations, thanks to two phases. First the 3D template is reconstructed using rigid SfM (Shape-from-Motion) while the surgeon is exploring – but not deforming – the peritoneal environment. Second the 3D template is used during surgery to infer a quasi-conformal deformation to the current 3D shape from a single laparoscopic image only. This makes both sequential processing and effective self-recovery from tracking errors possible. The proposed approach has been validated on in-vivo laparoscopic videos of the abdominal wall and a uterus. Experimental results illustrate the ability of our method to deal with extensible deformations of the tissues.